The present invention relates generally to the field of software development and testing, and more particularly to identifying effective test suites for software performance testing.
Software testing may involve two aspects: functional testing, and performance testing. Functional testing, more commonly referred to simply as “debugging,” tests that the software properly performs all design functionality. Functional testing is typically performed by executing functional test suites that attempt to exercise every execution path in the software to determine that each path executes correctly with the desired results.
Performance testing assumes that the software functions properly, and attempts to identify software performance issues, or performance antipatterns, which are often related to the physical environment in which the software executes. Performance antipatterns may include reduced throughput, reduced response time, software crashes, etc. Performance antipatterns may often be identified by exercising the software in resource constrained environments. For example, a software application is exercised in an environment in which memory resources available to the application are purposely reduced. In this environment, the application exhibits performance antipatterns, such as unacceptable response time, or crashing. This may indicate that the application, properly or improperly, requires excessive memory resources. An analysis may be performed to identify the cause of the excessive memory usage, and the software may be refactored to address the antipattern. Similar to functional testing, performance test suites may be executed that create particular environments in which certain resources, individually or in combination, are purposely constrained or made available. Each such environment may be referred to as a context, and aspects of the test environment accessed or used by the test suite may be referred to as context parameters.